Design and Applications of a Secure and Decentralized Distributed Hash Table

Distributed Hash Tables (DHTs) are a powerful building block for highly scalable decentralized systems. They route requests over a structured overlay network to the node responsible for a given key. DHTs are subject to the well-known Sybil attack, in which an adversary creates many false identities in order to increase its influence and deny service to honest participants. Defending against this attack is challenging because (1) in an open network, creating many fake identities is cheap; (2) an attacker can subvert periodic routing table maintenance to increase its influence over time; and (3) specific keys can be targeted by clustering attacks. As a result, without centralized admission control, previously existing DHTs could not provide strong availability guarantees. This dissertation describes Whānau, a novel DHT routing protocol which is both efficient and strongly resistant to the Sybil attack. Whānau solves this long-standing problem by using the social connections between users to build routing tables that enable Sybilresistant one-hop lookups. The number of Sybils in the social network does not affect the protocol’s performance, but links between honest users and Sybils do. With a social network of n well-connected honest nodes, Whānau provably tolerates up toO(n/ logn) such “attack edges”. This means that an attacker must convince a large fraction of the honest users to make a social connection with the adversary’s Sybils before any lookups will fail. Whānau uses techniques from structured DHTs to build routing tables that contain O(√n logn) entries per node. It introduces the idea of layered identifiers to counter clustering attacks, which have proven particularly challenging for previous DHTs to handle. Using the constructed tables, lookups provably take constant time. Simulation results, using large-scale social network graphs from LiveJournal, Flickr, YouTube, and DBLP, confirm the analytic prediction that Whānau provides high availability in the face of powerful Sybil attacks. Experimental results using PlanetLab demonstrate that an implementation of the Whānau protocol can handle reasonable levels of churn. Thesis Supervisor: M. Frans Kaashoek Title: Professor of Computer Science and Engineering